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A project was designed to evaluate TFiAH, a math 
anxiety reduction program for undergraduate students preparing to be 
elementary school teachers. The program consisted of two main 
components: (1) instruction intended to improve problem solving 
skills using the areas of patterns, probabilityr measurement r 
approximationr and estimation; and (2) activities designed to reduce 
math anxiety and to increase confidence in mathematics. The outcome 
evaluation for a two-year materials development project consisted of 
replications of small-scale trials of the materials. Outcome measures 
included both attitudinal and math concept instruments. Future 
evaluation research in this * or similar programs can concentrate in 
several areas arising from the development and evaluation of the TEAM 
materials and course. Areas such as the instructional approach, the 
integration cf testing and teaching, teacher confidence and 
competence, and interaction between aptitude and treatment need more 
research. (EG) 
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Abstract 



TEAJI is a math anxiety reduction program for undergraduate students 
preparing to be elementary school teachers. The program can be offered as 
a course preparatory to entering the teacher education sequence, and con- 
sists of two main components: 1. instruction designed to improve problem 
solving skills using the areas of patterns, probability, measurement, and 
approximation and estimation; 2. activities designed to reduce math anxiety, 
and increase confidence in mathematics. The outcome evaluation for the 
two-year materials development project consisted of replications of small- 
scale trials of the materials and was conducted as part of a VJEEA grant. 
Outcome measures included both attitudinal and iri^th concept instruments. 

Issues for evaluators of this or similar programs are in the measure- 
ment and design areas. Specifically, issues arise in the selection or 
development of ins'truments to evaluate math achievement and problem 
solving, and the use of open-ended instruments which provide protocols for 
.content analyses. Design issues arise from difficulties in selecting com- 
parison groups and the combination of attitudinal (conf?dence or anxiety 
reduction) components within a mathematics course. The latter issue leads to 
considerations of ATI questions of the combination of components best suited 
to individuals with different levels of initial math anxiety and math 
knowledge. 
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Paper presented at the annual meeting of the American Educational 
Research Association, Los Angeles, April 1981, as part of the symposium. 
Designing Evaluations for Math Confidence and Competence Programs: Current 
Status and Needed Developments. 
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Issues in the Outcome Evaluation of a Math Anxiety Reduction Program 

For Teacher Education Students 

TEAM is a math anxiety reduction program for undergraduate students 
preparing to be elementary school teachers (Chapline and NeA-mian, 1980). 
The program can be offered as a course preparatory to entering the teacher 
education sequence, and consists of tv/o main conponents: 1. instruction 
designed to improve problem solving skills using the areas of patterns, 
probability, measurement, and approximation and estimation; 2. activities 
designed to reduce math anxiety, and increase confidence in mathematics. 
The outcome evaluation for the two-year materials development project con- 
sisted of replication of small-scale trials of the materials. Outcome 
measures included both attitudinal and math concept instruments. 

The major findings from the outcome evaJ nation are reported here, 
along with the strengths and weaknesses of the evaluation and the issues 
which future evaluations should consider. Formative evaluation of the 
program has been reported by Chapline (1981). 
1. Major findings 

The goals of TEAM were to increase mathematics knowledge and 
understanding, increase confidence in mathematics, reduce math anxiety, 
increase the perception of math as a female domain, and counteract sex 
bias in the classroom. Three groups of students participated in TEAM 
during the two years of the program. During the first year, TEAM I 
students (N = 44) participated in a one-semester course in which the 
instruction vras carried out by a math educator and a psychologist. During 
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year 2, the materials were used in two colleges in the fall, vlth a math 
educator and a psychologist in one setting and a mathematician and a 
psychologist in the second setting (TEAM II, N =3^0 • In the spring of the 
second year the course was taught by a math educator with a "back-up" (non- 
instructional settings) of a psychologist (TEAM III, N = 29). 

Although the evaluation design differed somewhat for each phase of 
TEAM, the following outcome-related elements were included in the evaluation 
designs: 

A pre-course and post-course administration of three mathematics 

attitude scales: The Mathematics Anxiety Rating Scale (MARS, 

Richardson and Suinn, 1972) and the Fennema/Sherman Confidence 

and Usefulness Scales (1976); 

A pre-course and post-course questionnaire; 

Comparison group data on attitudes for TEAMS I and III; and 

A pre-course and post-course measure of mathematics achievement 

which was modified after TEAM I. 

Two of the attitude measures used, the MARS and the Fennema/ Sherman 
Confidence Scale were positively correlated (r = .37; Chapline, et al., 
1979). The MARS is a 98 item instrument which requires the respondent 
to rate a situation on how frightening it is. The response categories of 
this five point Likert-type scale range from "not at all to "very much." 
Situations include dividing a luncheon bill, taking a math test, seeing 
someone use a slide rule, or meeting your math professor. Our experience 
is that the MARS requires 30 to 60 minutes to administer, depending on the 
reading level of the respondents. The first factor in a factor analysis of 
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the mRS (Rounds and Hendel. 1979; DonKer and Tittle. 1980) appears to be a 
test anxiety factor and accounts £or approximately 507. ot the variance. 

The Fennema/Shorwn Confidence Scale does not appear to be as heavily 
reliant on test anxiety. The twelve items include, a.,ong others. "I an, 
sure I can learn mathematics": "I don't believe 1 can do advanced mathematics" 
"I'm not the type of person who does well in math"; "I'm good in math." 
These are rated on a five point Likert-type scale ranging from highly agree 
to highly disagree. The Fennema/Sherman Scales require 15-20 minutes to 
administer. 

comparison group data were obtained for TEAMs"! and III. The comparison 
groups were mainly students beginning the teacher education sequence. 
Results on the attitude measures indicate the effect of the use of TE»! 
materials. Table 1 represents the results for the MARS and the Fennema/ 
Sherman Confidence Usefulness scales. There are consistent and significant 
decreases in the «ARL^cores. Decreases for the Comparison III group were 
not Significant. Analysis of covariance for TEAM III,a_nd Comparison III 
groups, using the MARS pretest as the cc .ate. showed F for the covariate 
as 62.8 (P - .000) and the main effect f.. groups significant (F = 13.86. 
p . .0004). «hen adjusted for initial status on the IIARS scores the post- 
test difference in MARS scores is significant, with the TEAM III group 
showing a lower mean score on the measure of math" anxiety. Figure 1 shows 
the (unadjusted) pre-post means for TEAM III and Comparison III groups. 
The change in the TEAM group's mean does not appear to be due entirely to 
statistical regression, although the decreas^e fo^r MARS scores for the 
comparison group III probably is (as Tryon L1980J notes, however, a wide 
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variety of treatments seems to be effective in reducing self-reported test 
anxiety; perhaps, for math anxiety also, the fact that any treatment is attempted 
is the critical factor). 

Ti7o of the TEAM groups show significant increases pre^-post in ratings 
using the Fennema /Sherman Usefulness and Math Confidence scales. TEAM. II 
does not show significant increases, but the trend is the same direction. 
The Usefulness scale tended to be significant in the expected direction 
although the actual mean scores increased less than five points of a pos- 
sible 60. High usefulness scores were obtained on the pre-course measure, 
with a resultant ceiling effect with this sample. (The Usefulness scale may 
be an potential predictor Of project drop-outs. In a voluntary workshop- 
type setting, preliminary dr.ta indicate that those students who are anxious 
and see math as useful may be motivated to remain in programs; for those 
who do not perceive math as useful, the anxiety produced by the course is 
not offset and students remove themselves from the situation [Bates, 1981J.) 

Numerous problems v/ere encountered in assessing the changes in 
mathematics achievement. The project-developed concepts test was not linked 
closely to the TEAM content nor to the major objective of teaching students 
strategies for problem solving: breaking a problem down into smaller units, 
trying several approaches to solving mathematical problems, and compiling 
data in an organized manner so that patterns can be identified. On the 
concepts test which was developed, some students received high pre-course 
scores and this ceiling effect complicated the analysis of the results. (A 
post-test only comparison of TEAM I and Comparison I groups showed- no 
difference; for TEAM III, the post-test mean was 5.2 points above the 
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pre-test mean and this difference was significant,) 

The pre- and post-course questionnaires were found to be useful 
instruments, providing a rich data source. For example, the majority of 
TEAM students reported that there had been a positive change in their 
attitudes toward i^athematics. Overall, less than 20% reported no change 
and only 2% (two students) reported attitudes a little more negative. Of 
considerable interest were the open responses to the question. Do you 
think your attitudes toward math have changed any this semester? How? 
Sample responses given by TEAM I students included the following: 

1, Oh Yes! In the beginning I didn't realize that I had certain 
abilities. Now I realize I have the potential, 

2, Yes, I don't find it as hard now, Upped my confidence, I 
was very scared in the beginning because the last math class 
(in high school) was awful, 

3, I do, I am more patient. Before if I didn't understand it, 
iM skip it;».*now I don't give up too easily, 

4, I felt v.^ry relaxed in the class, I see that you can experiment 
with math more. It's interesting. 

5, Yes, I don't feel as incompetent as I used to. I feel I have 
a. better understanding of math and I don't feel as helpless, 

'6, Yes, definitely. I previously saw numbers in any form and I 
blocked them out. Now when I see numbers 1 take time, I 
concentrate, 

7, I'm a little more comfortable with it. With enough help and 
work I can do better, I just got a job where math is involved. 

2. Strengths and Weaknesses 

The question of comparison groups continues to plague the evaluator. 

Since most of the TEAM participants were a self-selected sample of course 

registrants, the similarity of any comparison groups is questionable. The 

waiting list approach was attempted although the N was limited to seven.' 
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Pre-semester measures were mailed to homes and returned by mosu students on 
the promise of special workshops for uo-credit after the semester. Post- 
semester measures, however, were not returned although numerous attempts were 
made to obtain these data; students were no longer interested in the workshops 
once the semester ended. 

In addition to the self--selection problem, comparison classes were 
difficult to identify which held constant the vocational preference 
(elementary education) and the number of years of college education. A third 
problem with control group data in a field setting is the reliance on con- 
tacts with other faculty to request the administration of tests to their 
class and the use of instructional time for data collection. 

Another weakness of the evaluation data was the measure of math 
achievement test that "fit" the objectives. The math concept test used by 
the project included some of the concepts in the math content. However, it 
did not focus on nor was the scoring such that it was possible to identify 
increases in skills in applying problem solving strategies. In this respect, 
the weakness is similar to that, of studies reviewed by Tryon (1980) . She 
sunimarized her review by pointing to the changes in self-reported test 
anxiety with almost any treatment; but, "Changing the student's academic 
performance is another matter" (p. 364). By the end of the project we were 
clear about the specific type of instrument we neeaed, but did not have the 
necessary protocols to construct such a score or measure. That students 
perceived the attainment of such a goal is clear in some of their comments, 
cited above. 

8 • 
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A major strength of the outcome evaluation design, however, was the 
replication of the program ideas 'at different sites, using different 
iTistructors, and the different ways in which the psychologist and math 
educator or mathematician worked together. The two colleges involved are 
both large urban colleges, pert of a university system, with diverse racial 
and ethnic groups of students. 

3. Content Issues Raised by the Outcome Evaluation 

There are several issues for future evaluation research arising from 
the development and evaluation of the TEAM materials and course. These are 
in the areas of the instructional approach, the integration of testing with 
teaching, the relationship between the developmert of confidence and com- 
petence in a subject area, the measurement of problem solving skills, and 
the effect of. varying treatment according to initial characteristics of 
participants. 

The instructional approach . The philosophy underlying the teaching 
strategy used in the TEAM materials is that the confidence and knowledge of 
the learner are best enhanced by the use of an inductive approach. In this 
approach, learners are not given the problem solving rules immediately, but 
arrive at them through generalizing from the solutions to a number of care- 
fully selected and sequenced problems. As indicated by the student logs and 
open-ended responses, the approach was fruitful in terras of the learners' 
ability to reconstruct forraulas, to feel "comfortable" during examinations, 
and to feel more confident about math. For some students, however, this is 
a new approach and their past experience makes them feel that the approach 
is lengthy and drawn out. It will be important to determine if the approach 
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is effective for different types of students. (This also relates to the 
raeasuremer.t problem discussed below.) 

The integration of testing with teaching . It has been suggested that 
a key variable in assisting learning in mathematics is the frequent use of 
s^or.t quizzes (Begle, 1979). In the TEAM project, tests are used as an 
opportunity to explore math--related anxiety, kniile there is some research 
on the use of tests to increase learning, there is little or no research 
that was located on frequent uses of tests vitbin a program in which one of 
the goals is the reduction of anxiety. The number of quizzes needed within 
programs in which the instructional goals include both competence and 
reduction of content-related anxiety is another issue ,for evaluation research. 

Confidence and competence . The TEAM materials and course have as their 
goal increasing mathematics competency and simultaneously building confidence 
in one^s ability to use mathematics, with nn accompanying decrease in anxiety 
about mathematics. The interaction of competence and confidence is one of 
the as yet untesced hypotheses of TEAM. To achieve this goal TEAM materials 
were designed to present primarily unfamiliar mathematical concepts, so 
students could perceive clearly their gains in knowledge of mathematics and 
so that prior negative experiences with particular content areas would not 
have a major effect. Another issue for evaluation research, then, is the 
use of unfamiliar mathematical concepts to increase confidence (as opposed 
to the use of familiar mathematical concepts) in programs such as TEAM. 
That is, does an imaginative and innovative approach to old skills facilitate 
the reduction of math anxiety, as well as increasing mathematical problem 
solving competencies? 
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The measurement of problem solving skills . Since the inductive approach 
to teaching mathematical concepts is intended to give the learner a new 
approach to problem solving in mathematics, the measurement of problem 
solving skills is critical. Most measurement of problem solving examines 
problem outcomes or solutions, but does not assess the most important com- 
ponent of the skill, the process by which the student solves the problem. 
The TEAM project had anticipated working on the measurement problem, but 
did not have the resources to do so within the time constraints of the 
project. A serious need to be met is the development of a process measuring 
"instrument." Careful examination cf the effects of the instructional 
approach, as well as the interdependence of confidence and competence, is 
not possible without further development of the measurement instrument. 
Research is neaded to define the problem solving "process" in more detail 
{^2oT these adult learners). Clarification of the initial status in terms of 
how adult students approach math problems and definitions of the desired - 
process outcome are required. Some work on the measurement process is 
reported, in Malone, ^ aJ.. (1980) and Schoen and Oehmke (1980), and may 
serve as a model for projects such as TEAM. 

The interaction between aptitude and treatment . TEAM participants 
were recruited by informing students that the program existed and urging 
them to enroll. This resulted in a self-selected sample of students who 
had enrolled for various reasons and who had different levels of mathematical 
skills and anxieties. It is expected that students with low skill and high 
anxiety need different treatments than students with high skill and high 
anxiety or low skill and low anxiety. Systematic identification of these 

11 



Issues in the Outcome 



10 

subgroups is needed in evaluation studies (Tobias and Weissbrod, 1980). 

Research on test anxiety also provided further impetUvS to identify 
subgroups of students for whom different ^'treatments" may be appropriate, 
Tryon's (1980) review of test aiixiety identified two components of test 
anxiety, a worry and an emotionality component. The testing situation 
(which is a part of math confidence development in the TEAM program) is 
presently thought to evoke both learned task drives and learned anxiety 
drives. Some anxiety drives may be task relevant while others are task 
irrelevant. Performance is facilitated by those learned task drives and 
learned anxicity drives that are task relevant; task-irrelevant anxiety 
drives decrease task performance, A high test-anxious person probably 
attends to both self-relevant and task relevant variables and worries 
during examinations (or test-like classroom situations). The low test- 
anxious person is attending mainly to the task. Informal observations of 
the first TEAM group supports this interpretation in part. An extreme 
form of such behavior ce 'Id *"ound; during discussions in which students 
prepared for exarlnations, they were observed carrying out such self- 
relevant tasks as taking out a mirror and combing their hair. These 
students were not making task relevant responses, and such behavioral 
indicators may provide a diagnostic basis other than a self-report inventory 
such as the MARS for identifying subgroups of students for whom different 
approaches, or a sequence of approaches, may be required to increase both 
confidence and performance in mathematics. .Tobias and V7eissbrod (1980) 
also sugf^ot the use of indicators other than paper and pencil measures. 

TO 
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In summary, the design of evaluations for math confidence and 
competence programs such as TEAM face many fundamental questions. Aware- 
ness of these questions can assist evaluators to relate new programs to 
the existing psychological literature on test anxiety and to the developing 
literature of mathematics competence and confidence. 
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Table 1 

• Pretest-Posttest means, SD's and _t for the MARS and F-S Scales: 

TEAM I, II, III 

MATHEMATICS ANXIETY RATING SCALE>'« 







Pretest Mean 


Posttest Mean 


N 


t 


P 


TEAM 


I 


245.3 (SD=64.7) 


211.6 (SD=64.6) 


44 


4.26 


.001 


TEAM 


II 


249.8 (SD=82.1) 


195.8 (SD=66.6) 


34 


3.13 


.003 


TEAM 


III 


260.5 (SD=69.1) 


198.1 (SD=57.5) 


30 


6.50 


.000 


Comparison III 


231.1 (SD=63.6) 


218.1 (SD=78.2) 


41 


1.74 


.089 






FENNEMA- SHERMAN 


MATH USEFULNESS-'"'* 








TEAM 


I 


34.8 (SD=6.3) 


38.1 (SD=5.7) 


44 


3.69 


.001 


TEAM 


II 


48.1 (SD--5.3) 


49.3 (SD=6.3) 


34 


.88 


ns 


TEAM 


III 


44.3 (SD=7.1) 


48.7 (SD=5.9) 


30 


4.20^ 


.000 






FENNEMA- SHERMAN 


MATH CONFIDENCEi^'* 








TEAM 


I 


20.5 (SD=9.6) 


26.4 (SD=9.3) 


44 


5.86 


.001 


TEAM 


IX 


35.9 (SD=11.3) 


41.0 (SD=11.5) 


34 


1.82 


ns 


TEAM 


III 


30.1 (SD=11.8) 


38.5 (SD=10.5) 


30 


7.60 


.000 



''^Higher scores indicate more math anxiety: 98 items, 1-5 points each. 



'^<*'fllighcr scores indicate more math usefulness or confidence: 12 items, 
1-5 points each. 
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